NAVAL 

POSTGRADUATE 

SCHOOL 

MONTEREY,  CALIFORNIA 


THESIS 


OPTIMIZING  BANDWIDTH  IN  TACTICAL 
COMMUNICATIONS  SYSTEMS 

by 

Criston  W  Cox  Jr. 

June  2005 

Thesis  Advisor:  William  Kemple 

Second  Reader:  John  Osmundson 


Approved  for  public  release;  distribution  is  unlimited. 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


REPORT  DOCUMENTATION  PAGE 


Form  Approved  OMB  No.  0704- 
0188 


Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per 
response,  including  the  time  for  reviewing  instruction,  searching  existing  data  sources,  gathering 
and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send 
comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  headquarters  Services,  Directorate 
for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA 
22202-4302,  and  to  the  Office  of  Management  and  Budget,  Paperwork  Reduction  Project  (0704-0188) 
Washington  DC  20503. 

I.  AGENCY  USE  ONLY  (Leave  blank)  2.  REPORT  DATE  3.  REPORT  TYPE  AND  DATES  COVERED 

June  2005  Master's  Thesis 

4.  TITLE  AND  SUBTITLE:  Optimizing  Bandwidth  on  Tactical  5.  FUNDING  NUMBERS 

Communications  Systems 

6.  AUTHOR (S)  Captain  Criston  W  Cox  Jr  _ 

7.  PERFORMING  ORGANIZATION  NAME (S)  AND  ADDRESS (ES)  8.  PERFORMING  ORGANIZATION 

Naval  Postgraduate  School  REPORT  NUMBER 

Monterey,  CA  93943-5000 

9.  SPONSORING  /MONITORING  AGENCY  NAME (S)  AND  ADDRESS (ES)  10 .  SPONSORING/MONITORING 
-3rd  Marine  Division,  III  MEF,  Okinawa  Japan  AGENCY  REPORT  NUMBER 

-Marine  Corps  Tactical  Systems  Support  Activity,  Camp 
Pendleton  California 

II.  SUPPLEMENTARY  NOTES  The  views  expressed  in  this  thesis  are  those  of  the  author  and  do  not 
reflect  the  official  policy  or  position  of  the  Department  of  Defense  or  the  U.S.  Government. 

12a.  DISTRIBUTION  /  AVAILABILITY  STATEMENT  12b.  DISTRIBUTION  CODE 

Approved  for  public  release;  distribution  is  unlimited. 

13.  ABSTRACT  (maximum  200  words) 

Current  tactical  networks  are  oversaturated,  often  slowing  systems  down 
to  unusable  speeds.  Utilizing  data  collected  from  major  exercises  and 

Operation  Iraqi  Freedom  II  (OIF  II)  ,  a  typical  model  of  existing  tactical 
network  performance  is  modeled  and  analyzed  using  NETWARS,  a  DISA  sponsored 
communication  systems  modeling  and  simulation  program.  Optimization 

technologies  are  then  introduced,  such  as  network  compression,  caching. 
Quality  of  Service  (QoS) ,  and  the  Space  Communication  Protocol  Standards 
Transport  Protocol  (SCPS-TP) .  The  model  is  then  altered  to  reflect  an 

optimized  system,  and  simulations  are  run  for  comparison.  Data  for  the 
optimized  model  was  obtained  by  testing  commercial  optimization  products  known 

as  Protocol  Enhancement  Proxies  (PEPs)  at  the  Marine  Corps  Tactical  Systems 

Support  Activity  (MCTSSA)  testing  laboratory. 


14.  SUBJECT  TERMS  15.  NUMBER  OF 

Protocol  Enhancement  Proxies,  compression,  caching,  Quality  of  PAGES 
Service,  bandwidth,  optimization,  modeling  and  simulation,  NETWARS ,  79 

TCP  over  Satellite,  tactical  data  networks,  tactical  networks.  ^^^^RICE^CODeT 

17.  SECURITY  18.  SECURITY  19.  SECURITY  20.  LIMITATION 

CLASSIFICATION  OF  CLASSIFICATION  OF  THIS  CLASSIFICATION  OF  OF  ABSTRACT 

REPORT  PAGE  ABSTRACT 

Unclassified  Unclassified  Unclassified  UL 

NSN  7540-01-280-5500  Standard  Form  298  (Rev.  2-89) 

Prescribed  by  ANSI  Std.  239-18 


1 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


11 


Approved  for  public  release;  distribution  is  unlimited. 


OPTIMIZING  BANDWIDTH  OF  TACTICAL  COMMUNICATIONS  SYSTEMS 


Criston  W  Cox  Jr 

Captain,  United  States  Marine  Corps 
B.A.,  The  Citadel  1999 


Submitted  in  partial  fulfillment  of  the 
requirements  for  the  degree  of 


MASTER  OF  SCIENCE  IN  SYSTEMS  TECHNOLOGY 
(JOINT  COMMAND,  CONTROL,  COMMUNICATIONS,  COMPUTERS  AND 

INTELLIGENCE  ( JC4I) ) 


from  the 


NAVAL  POSTGRADUATE  SCHOOL 
June  2005 


Author:  Captain  Criston  W  Cox  Jr. 


Approved  by:  Dr.  William  Kemple 

Thesis  Advisor 


Dr.  John  Osmundson 
Second  Reader/Co-Advisor 


Dr.  Dan  Boger 

Chairman,  Department  of  Information  Sciences 


in 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


IV 


ABSTRACT 


Current  tactical  networks  are  oversaturated,  often  slowing 
systems  down  to  unusable  speeds.  Utilizing  data  collected  from  major 
exercises  and  Operation  Iraqi  Freedom  II  (OIF  II) ,  a  typical  model  of 
existing  tactical  network  performance  is  modeled  and  analyzed  using 
NETWARS,  a  DISA  sponsored  communication  systems  modeling  and  simulation 
program.  Optimization  technologies  are  then  introduced,  such  as 
network  compression,  caching.  Quality  of  Service  (QoS) ,  and  the  Space 
Communication  Protocol  Standards  Transport  Protocol  (SCPS-TP) .  The 
model  is  then  altered  to  reflect  an  optimized  system,  and  simulations 
are  run  for  comparison.  Data  for  the  optimized  model  was  obtained  by 
testing  commercial  optimization  products  known  as  Protocol  Enhancement 
Proxies  (PEPs)  at  the  Marine  Corps  Tactical  Systems  Support  Activity 
(MCTSSA)  testing  laboratory. 
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EXECUTIVE  SUMMARY 


Command  and  Control  (C2)  techniques  and  procedures 
radically  changed  with  the  emergence  of  Net-centric 
capabilities.  In  just  a  ten  year  period,  the  author 
witnessed  a  complete  revolution  in  Division  level  C2 .  In 
1993,  Radio  operators  hand-delivered  messages  to  staff 
officers,  who  plotted  points  on  individual  wall-sized  map 
boards.  Information  sharing  meant  walking  over  to  the 
other  staff  officer's  cubicle  and  telling  him  what  was 
deemed  necessary  for  him  to  know.  Today,  information  is 
more  centralized,  enabling  it  to  be  efficiently  shared, 
filtered,  and  accessed  as  needed.  C2  systems  are  displayed 
via  projectors  or  plasma  televisions,  and  can  be  displayed 
and  sorted  via  overlays  for  specific  views  on  the  staff 
officer's  laptop.  Instead  of  the  Commanding  General 
walking  around  to  each  section  for  updates,  he  can  sit 
before  one  screen,  while  each  staff  officer  briefs  from  the 
same  Common  Operational  Picture. 

The  revolution  of  netcentric  C2  has  been  effectively 
realized  by  both  the  war  fighting  staffs  and  the 
communicators  who  provide  the  networks  on  which  these 
capabilities  reside.  However,  the  communications  community 
has  not  yet  grasped  standardized  methods  for  optimizing  the 
systems  to  get  the  most  out  of  them.  This  thesis  analyzes 
various  methods  designed  to  optimize  tactical 
communications  systems  and  provide  significantly  higher 
levels  of  service  with  currently  fielded  systems. 
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I.  THE  PROBLEM  WITH  CURRENT  USMC  TACTICAL  NETWORKS 

A.  BACKGROUND 

The  Unites  States  Marine  Corps  has  made  a  tremendous 
transformation  in  the  way  the  Combat  Operations  Center 
(COC)  is  operated,  and  how  commanders  command  and  control 
(C2)  the  battlefield.  The  COC  is  operated  of  the 

Commander's  principle  staff,  and  serves  as  the  nucleus  of 
his  C2  functions.  Up  to  the  early  to  mid  1990' s,  the 
primary  means  of  communications  in  the  COC  was  single 
channel  radio.  Junior  enlisted  Marines  transcribed  radio 
messages  onto  a  "yellow  canary"  form,  and  promptly  handed 
it  off  to  the  staff  officer  for  action.  Each  section  used 
its  wall-sized  map  boards  to  maintain  situational  awareness 
by  marking  positions  with  color-coded  tacks.  Information 
sharing  meant  that  a  staff  officer  was  obligated  to  walk 
over  and  deliver  the  information  personally. 

Today,  technological  implementations  have  radically 
changed  COC  operations.  Flat  screen  42-inch  plasma 


monitors 

and 

large 

screen  projectors  provide 

visual 

support. 

such 

as 

a 

near  real-time  Common  Operational 

Picture 

(COP) 

of 

the 

Global  Positioning  System 

(GPS) 

equipped  units  as  they  move  across  the  battlefield. 
Situational  Awareness  (SA)  is  provided  by  a  multitude  of 
systems,  to  include  a  web-enabled  database  that  is  fed  by 
higher,  adjacent,  and  supporting  units  as  they  report 
events.  At  the  Regimental  level  and  above,  data  systems 
have  become  the  primary  means  of  communications,  providing 
e-mail,  C2  systems,  and  web-based  applications.  Even  the 
highly-mobile  infantry  battalions,  which  still  rely 
primarily  on  single  channel  radio  with  limited  data  while 
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on  the  move,  are  becoming  more  and  more  reliant  on  data 
systems  in  the  Battalion  COC  once  in  a  static  position. 
For  example,  just  prior  to  retaking  Fallujah  in  November 
2004,  two  of  the  three  infantry  battalions  observed 
operating  in  Fallujah  and  Ramadi  had  saturated  their  256 

kbps  SIPRNET  (Secure  Internet  Protocol  Routed  Network)  at 
98  percent  utilization.  The  radio  remote  units  associated 
with  single  channel  radio  nets  are  barely  apparent  above 

the  Regimental  level  COC,  almost  a  defunct  byproduct  of 
days  gone  by. 

B.  THE  LEARNING  CURVE 

All  of  the  applications,  email,  and  C2  systems  that 
Marines  so  heavily  depend  upon  in  the  field  ride  over  the 
network,  which  is  designed,  built,  and  managed  by 
communications  Marines.  The  Marine  Corps  as  a  whole  has 
endured  a  steep  learning  curve  over  the  past  decade  to 

embrace  technological  advances,  but  none  more  so  than  the 
communications  Marines.  Each  time  they  deploy  to  the 

field,  they  reprogram  routers,  rebuild  the  Exchange 
servers (mail ) ,  Domain  Name  Servers  (DNS),  and  domain 
controllers,  create  and  manage  all  user  accounts,  put 
user' s  computers  on  the  network,  and  manage  the  Deployed 
Security  Interdiction  Device,  an  Intrusion  Detection  System 
(IDS)  and  firewall  system.  Data  communications  Marines  are 
good  at  what  they  have  been  trained  to  do,  but  their 
training  is  limited  in  scope.  They  are  not  taught  how  to 
manage  Quality  of  Service  (QoS)  features  of  our  existing 
Cisco  routers,  such  as  Priority  or  Custom  Queuing 
strategies.  Layer  2  (of  the  Open  Systems  Interconnection 
model  (OSI  model) )  management  of  switches  generally  does 
not  happen,  and  the  Local  Area  Networks  (LANs)  behind  the 
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firewall  are  treated  as  plug  and  play  systems.  This  thesis 
analyzes  various  network  optimization  technologies  that  the 
USMC  could  implement  to  more  proactively  manage  these 
systems  and  maximize  the  limited  bandwidth. 

C.  THE  PROBLEM  AND  PURPOSE 

Tactical  networks  are  becoming  increasingly  saturated 
with  users  and  applications  as  the  need  for  information 
exceeds  the  capabilities  of  the  systems  that  provide  it. 
"At  the  peak  of  [OIF  I],  DISA  (Defense  Information  Systems 
Agency)  claimed  that  3  Gbps  of  satellite  bandwidth  was 
being  provided  to  the  theater,...  30  times  the  bandwidth  made 
available  during  Desert  Storm."1  A  typical  satellite 
connection  to  a  Standardized  Tactical  Entry  Point  (STEP)  is 
1024  kbps.  This  is  multiplexed  to  provide  Defense 

Information  System  Network  (DISN)  services2  to  many  tactical 
users.  After  being  multiplexed/demultiplexed  into  separate 
services,  the  data  networks  only  receive  a  portion  of  the 
bandwidth,  typically  less  than  384  kbps.  To  put  this  in 
perspective,  anywhere  from  50  to  more  than  100  users  may  be 
using  a  circuit  that  has  the  bandwidth  equivalence  of  a 
residential  Digital  Subscriber  Line  (DSL)  connection.  In 
addition,  due  to  the  adverse  effects  that  high  bit  errors 
and  latency  associated  with  satellite  transmissions  cause 
for  TCP  (Transmission  Control  Protocol) ,  actual  traffic 
throughput  capability  is  even  less  than  the  allotted 
bandwidth.  (The  problems  with  TCP  over  Satellite  will  be 
discussed  further  in  Section  III)  .  Given  the  crucial  role 

1  Leland  Joe,  Isaac  Porche  III,  Future  Army  Bandwidth  Needs  and 
Capabilities,  p.  11,  RAND  Corporation,  2004. 

2  DISN  services  normally  extended  to  the  Division  COC  include: 

Digital  Truck  Group  (DTG)  for  secure  and  nonsecure  telephone  service 

from  the  Defense  Switched  Network  (DSN) ,  Secure  Internet  Protocol 
Router  Network  (SIPRNET)  and  Unclassified  but  Sensitive  Internet 
Protocol  Router  Network  (NIPRNET) ,  and  Video  Teleconference  (VTC) . 
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communications  systems  play  in  maintaining  information 
superiority  and  commanding  and  controlling  the  battlefield, 
bandwidth  has  become  a  critical  asset  requiring  deliberate 
management  techniques. 

1 .  The  Problems  with  Current  Methods  for  Meeting 
Bandwidth  Requirements 

Senior  communicators  recognize  the  impact  of  growing 
user  requirements,  and  are  confronting  the  demands  with 
various  COTS  solutions  and  technologies,  from  simply 
leasing  more  bandwidth  to  various  WAN  acceleration 
products.  Moreover,  acceleration  products  are  often 
incompatible  between  vendors  due  to  proprietary  standards, 
causing  interoperability  issues  when  units  purchase 
different  optimization  solutions.  For  example,  during  OIF 
I,  MARCENT  purchased  SkyX  accelerators  to  terminate  an 
Intel  link  between  3rd  Marine  Air  Wing  and  1st  Marine 
Expeditionary  Force  (I  MEF) 3,  the  24th  Marine  Expeditionary 
Unit  (MEU)  purchased  Expand  accelerators,  and  the  STEP 
sites  implemented  the  ComTech  Turbo  IP  Accelerators  (I  MEF 
also  had  Expand  Accelerators) .  Each  of  these  products  used 
different  protocols  at  that  time,  which  were  not 
interoperable  and  could  only  be  used  on  internal  point  to 
point  links  with  like  devices  on  each  end.  Despite 
significant  performance  increases  observed  on  internal 
links,  links  to  adjacent  units.  Joint  Task  Force  (JTF) 
elements,  and  the  STEPs  remained  congested  due  to 
incompatible  proprietary  standards.  If  everyone  were  using 
a  standard  solution,  the  whole  system,  rather  than  a  few 
selected  links,  would  perform  more  efficiently. 


3  EMAIL:  LtCol  Mark  Bryant.  FW :  Request  For  Information 

BryantMH@mcsc . usmc . mil  October  13,  2004 


4 


DISA  has  established  the  Space  Communications  Protocol 
Standards  -  Transport  Protocol  (SCPS-TP) ,  ( MIL-STD-2045 - 

44000  and  Consultative  Committee  for  Space  Data  Systems 
CCSDS-714 . 0-B-l)  as  the  only  approved  form  of  TCP 
acceleration  for  mitigating  the  deficiencies  due  to  the 
satellite  link,  but  they  should  also  provide  further 
guidance  on  the  use  of  approved  compression  and  caching 
technologies,  which  would  provide  far  greater  throughput 
than  SCPS-TP  alone.  In  absence  of  such  guidance,  unit's 
requests  to  use  compression  or  caching  technologies  such  as 
those  featured  on  SkyX  and  Expand,  are  denied.  According 
to  the  MARCENTCOM  G-6  Operations  Officer  during  OIF  I: 

Sky-X  boxes  were  purchased  by  MARFORPAC/MARCENT 
(i.e.  I  signed  the  purchase  request) .  It  was  our 
hope  that  we  could  use  them  ISO  our  GMF  links 
into  the  STEPs,  but  DISA  shot  us  down  -  they  did 
not  like  the  caching  done  by  the  Sky-X, 
especially  when  we  were  pulling  SIPR  traffic  from 
them.  So,  we  only  used  on  a  pt-pt  KU  shot  (8 
mbps)  between  the  Wing  and  I  MEF  .  .  .  That  was 
our  means  for  passing  raw  intel  taken  from  the 
FA-18 's  to  the  I  MEF  intel  cell  at  Camp  Commando...4 

The  caching  aspect  should  not  be  treated  any 
differently  than  the  stored  data  on  a  SIPRNET  server  or 
client's  hard  drive.  Once  an  accelerator  touches  a 

classified  network,  it  should  simply  be  treated  with  the 
same  security  precautions  and  standards  as  we  do  any  other 
devices  placed  on  the  SIPRNET,  such  as  the  popular  mini¬ 
thumb  USB  hard  drives  so  commonly  used  in  Iraq. 

Another  solution  to  increasing  bandwidth  is  to  simply 
buy  more,  which  has  proven  to  be  inordinately  expensive 
comparative  to  the  cost  of  achieving  comparable  data  rates 

4  Email:  LtCol  Mark  H  Bryant,  SUB J :  RE:  FW:  Request  For  Information, 
January  20,  2005.  mark . h . bryant@usmc . mil 
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by  maximizing  our  existing  systems  with  the  WAN 
acceleration  technologies  demonstrated  in  this  thesis. 
MARCENT's  Universal  Need  Statement  (UNS)  to  acquire  8Mbps 
Deployed  Ku-band  Earth  Terminal  (DKET)  and  Fly-away  Ku-band 
Earth  Terminal  (FKET)  terminals  and  supporting  Technical 
Control  Facility,  estimated  procurement  costs  at 
$10,762,000,  not  including  the  1.2  million  paid  yearly  for 
each  Ku  circuit's  bandwidth.5  I  MEF  also  procured 
additional  bandwidth  at  an  estimated  12  million  dollars  per 
year  for  their  five  commercial  Ku  band  links.6  One  goal  of 
this  thesis  is  to  demonstrate  how  comparable  data  rates  can 
be  achieved  at  a  fraction  of  the  monthly  cost  of  a  single 
leased  commercial  line  by  purchasing  COTS  optimization 
products,  which  could  be  used  on  all  future  exercises  and 
operations . 

Until  Headquarters  Marine  Corps  (HQMC)  provides 
further  guidance,  G-6  officers  can  only  influence  policy 
and  purchasing  decisions  within  their  scope  of 
responsibility.  The  effort  of  this  thesis  is  to  provide  an 
overview  of  the  various  techniques  and  technologies 
available,  provide  performance  and  interoperability 
perspectives,  and  demonstrate  network  optimization  metrics 
through  modeling  and  simulation. 


5  Email:  LtCol  Mark  H  Bryant,  SUB J :  RE:  FW :  Request  For  Informationr 
March  3,  2005.  mark . h . bryantOusmc . mil 

6  Email:  Capt  Billy  Cornell,  SUB J :  RE:  Request  For  Information,  March 
3,  2005.  Capt  Cornell  was  the  I  MEF  G-6  Operations  Officer  during  OIF 
I.  His  cost  are  estimated  as  follows:  (4)  Ku  Satellite  Terminals  using 
the  Intelsat  Constellation:  1 . 2M  per  for  6  months  of  access  (total  of 
28  Mbps  Bandwidth,  two  terminals  with  8Mbps,  and  two  with  6Mbps)  x2  for 
one  year  Total  =  9.6M.  A  fifth  Ku  terminal  at  Marine  Logistics  Command 
cost  an  additional  1 . 2M,  for  a  total  of  12  million. 
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Because  USMC  Communication  and  Information  Systems 
Officers  (0602)  are  the  target  audience,  the  remainder  of 
this  thesis  will  use  the  common  vocabulary  understood  by 
that  community  without  elucidation. 
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II.  CONDUCT  OF  THE  STUDY 


A.  DATA  COLLECTION 

In  support  of  this  thesis,  the  author  made  several 
Temporary  Additional  Duty  (TAD)  trips  for  additional 
training  and  to  observe  and  collect  data  from  tactical 
networks,  to  include: 

•  Exercise  Cobra  Gold  2004  (CG  04)  in  Thailand 

•  Ulchi  Focus  Lens  (UFL  04)  in  South  Korea 

•  Operation  Iraqi  Freedom  II  (OIF  II)  in  Iraq 

•  1st  Marine  Division  G-6  Office  and  the  Marine  Corps 
Tactical  Systems  Support  Activity  (MCTSSA)  at  Camp 
Pendleton  California 

•  The  Armed  Forces  Communications  and  Electronics 
Association  (AFCEA)  West  2004  Convention  to  attend  the 
Department  of  Defense  Architecture  Framework  (DODAF) 
Professional  Development  Course  in  San  Diego  Ca . 

•  Flagstaff  Arizona  to  assist  in  research  conducted  by 
Captains  Gilbert  Garcia  and  David  Joseforsky  on  the 
use  of  Free  Space  Laser  Optics  to  tie  in  voice  and 
data  from  antenna  hill  to  the  Unit  Operations  Center 
(UOC) . 

•  Booz  Allen  Hamilton' s  (BAH)  training  on  the  Network 
Warfare  Simulation  (NETWARS)  Program  at  the  DISA 
office  in  Washington  DC. 

•  Intense  Schools  Cisco  Certified  Networks  Administrator 
(CCNA)  and  Certified  Wireless  Network  Administrator 
(CWNA)  courses  San  Diego  Ca . 7 

Observations  from  these  experiences  are  referenced 
throughout . 


7  These  courses  were  funded  by  Mr.  Brian  Steckler,  NPS,  under  the 
Nemesis  Wireless  Warfare  Project.  All  other  funding  provided  by  MCTSSA 
and  3rd  Marine  Division. 
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1 .  Data  Collection  Utilities 

Three  utilities  were  employed  to  capture  network 
statistics  and  data.  The  model  discussed  in  Chapter  V 
represents  measures  of  performance  observed  using  these 
tools . 

a.  Network  Traffic  Analysis  System 

The  Network  Traffic  Analysis  System  (NTAS)  is  a 
Government  Of f-the-Shelf  (GOTS)  system  developed  using 
Commercial  Of f-the-Shelf  (COTS)  products.  NTAS  collects 
network  traffic  data  from  SNMP/RMON/RMON28  compatible 
network  devices,  NetFlow  data  from  Cisco  routers,  and 
network  configuration  data  from  routers  via  SNMP.  It 
employs  the  Net-Centric  concept  toward  the  collection, 
distribution,  and  access  to  network  performance  and 
topology  information  via  standard  data  transfer  and  Web 
client  interfaces9.  Unlike  other  software  used,  NTAS  is  a 
suite  of  applications  preconfigured  on  a  server,  requiring 
trained  personnel  to  install  and  maintain.  NTAS  equipment 
and  personnel  support  was  provided  during  CG  04,  UFL  04, 
and  OIF  II  by  DISA's  Cross-Program  Engineering  Branch 
(GE323) .  The  illustration  below  depicts  the  general 
concept  of  the  NTAS  server  both  collecting  traffic  and 
presenting  results  in  a  web-based  GUI  (Graphical  User 
Interface)  . 10  Because  the  NTAS  is  a  device  on  the  network, 
access  to  the  OIF  NTAS  can  be  granted  if  the  IP  address, 
username,  and  password  are  known. 


°  Simple  Network  Management  Protocol,  Remote  Monitor,  Remote  Monitor 
version  2 

9  ConOps  for  NTAS,  Cross-Program  Engineering  Branch  (GE323) 
GIG  Enterprise  Services,  Defense  Information  Systems  Agency 

-*-9  Illustration  taken  from  the  MARFORPAC  UFL  '03  COP  Analysis  Out- 
brief  Presentation,  Unclassified,  presented  19  November  2003  by  Major 
E.  Thomas  Powers,  former  NETWARS  Study  Lead. 


10 


NTAS 


Network  Traffic  Analysis  System 


Data  Flow 


Data  Collection  &  Analysis 


RMON  Data 


End-User 

Reports/Display 


Figure  1 .  NTAS  Collection  Scheme 

b.  SolarWinds 

SolarWinds  Engineer's  Edition  Version  7  is  a 
suite  of  networking  management  software  containing  Network 
Discovery,  Fault  Monitoring,  Performance  Monitoring  and 
Performance  Management  applications.11  SolarWinds  proved  to 
be  a  very  powerful  tool  for  network  analysis,  but  can  be 
dauntingly  technical  for  inexperienced  analysts.  This 
version  of  SolarWinds  better  serves  as  a  tool  for  analysis 
and  troubleshooting  than  for  network  monitoring  in  the 
sense  we  currently  employ  What's  Up  Gold  or  HP  Openview. 
What's  Up  Gold  generates  ICMP  traffic  on  the  network,  and 
uses  response  times  from  other  nodes  to  monitor  network 
health,  which  is  a  more  reactive,  rather  than  proactive 
approach  to  network  management.  SolarWinds  uses  Simple 
Network  Management  Protocol  (SNMP) ,  which  also  adds  traffic 


11 


http://SolarWinds.net/Tools/Engineer/index.htm]  January  05 
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to  the  network,  but  pays  far  larger  dividends  by  giving 
network  administrators  greater  ability  to  identify  problems 
before  the  network  goes  down,  instead  of  waiting  until  an 
icon  shows  up  red  to  begin  troubleshooting.  The  Engineers 
edition  used  featured  forty  seven  different  tools,  such  as 
Device  discovery,  which  provides  a  list  of  host  names  and 
IP  addresses  in  use,  circuit  utilization  percentage  graphs, 
errors,  and  transmit  and  receive  rates,  and  a  host  of 
Cisco,  DNS,  and  CPU  and  bandwidth  gauges  and  tools. 

c.  Ethereal 

Ethereal  Network  Protocol  Analyzer  is  a  free, 
open-source,  packet  capturing  utility.  Ethereal  passively 
captures  raw  data,  allowing  analysis  of  frames,  packets, 
and  protocols  traversing  the  network,  as  depicted  in  Figure 
2.  Ethereal  data  can  also  be  converted  into  Application 
Characterization  Environment  (ACE)  threads,  for 
visualizing,  analyzing,  and  troubleshooting  networked 
applications.12  ACE  threads  can  be  imported  in  NETWARS  to 
represent  actual  traffic  flows;  however  that  technique  was 
not  employed  for  this  study  due  to  the  classification  of 
the  RIPRNET  and  SIPRNET  traffic,  as  well  as  the  size  and 
complexity  of  the  network.  That  utility  was  not  discovered 
until  after  UFL .  Ethereal  was  employed  on  the  OIF  II 
network  to  determine  what  percentage  of  each  of  the  typical 
protocols  comprised  the  traffic.  The  results  observed 
influenced  the  background  traffic  loading  percentages  of 
the  model . 


12  Ace  User  Guide  for  IT  Guru,  OPNET  IT  GURU  Software  Documentation 
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Ethereal  Screen  Capture 


B .  STUDY  OVERVIEW 

To  demonstrate  the  level  of  service  being  provided  on 
current  networks,  a  model  of  a  typical  tactical  network  was 
constructed  using  NETWARS .  Designed  using  traffic  profiles 
and  circuit  utilization  percentages  observed  and  collected 
during  Cobra  Gold  04,  UFL  04,  and  OIF  II,  the  model  is 
capable  of  producing  statistical  data  by  simulating  traffic 
traversing  the  network.  Optimization  techniques  and 
technologies  are  then  researched  and  tested,  and  the  model 
is  modified  to  reflect  the  improvements  observed  during 
testing.  The  simulation  is  run  once  again,  and  the 
resulting  data  is  compared  to  the  previous  results  to 
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demonstrate  the  expected  benefits  of  a  layered  optimization 
approach.  The  NETWARS  model  and  results  are  presented  in 
Chapter  V. 
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III.  OPTIMIZATION  TECHNOLOGIES 


A.  COMPRESSION 

Compression  of  data  reduces  the  total  amount  of 
traffic  that  traverses  the  network,  thereby  reducing 
congestion.  Network  compression  affects  all  (previously 
uncompressed)  data  prior  to  transmission,  similar  to  the 
way  WinZip  reduces  the  size  of  individual  files  or  folders. 
It  is  common  to  see  vendors  advertise  60  to  80  percent 
compression  ratios,  and  according  to  the  independent 
Information  Technology  (IT)  consulting  company  META  Group, 
an  average  compression  ratio  of  60  percent  is  not  unusual. 
Meta  Group  provides  the  following  example  and  chart  to 
demonstrate  compression  rates  by  application:  "If  the  data 

destined  for  a  100%  utilized  1.5  Mbps  T1  link  is  60% 
reducible,  BCO  [Bandwidth  Compression  and  Optimization] 
could  instantly  reduce  traffic  contention  and  add  40% 
unutilized  capacity  for  peak  loads  or  new  applications. 
Assuming  additional  traffic  was  equally  compressible,  the 
equivalent  capacity  of  this  link  would  be  1/ (1  -  0.60)  x 

1.5  Mbps  =  3.75  Mbps."13 

Proprietary  compression  algorithms  require  a  like 
device  to  decompress,  but  open-source  algorithms  should  be 
interoperable  between  different  vendors. 


13  Peter  Firstbrook,  META  Group,  White  Paper,  Bandwidth  Compression 
and  Optimization:  Increasing  WAN  Capacity  and  Control  Without 
Increasing  Expenses,  pp .  6-7,  June  2003. 
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Figure  3.  Compressibility  of  Various  Applications  (From 

Ref.  13) 

B .  CACHING 

Caches  are  application-specific  proxies  that  store 
content  for  future  reference,  such  as  HTTP,  FTP,  DNS,  and 

streaming  media.  Data  accessed  across  the  network  is 
stored  locally  so  that  any  future  requests  from  users  of 

the  LAN  will  pull  from  the  cache  server  instead  of 
consuming  bandwidth  of  the  transmission  system.  Caching 
only  improves  performance  on  data  that  has  traversed  the 

network  and  been  stored,  so  the  benefits  of  caching 
increase  with  time  as  the  content  builds.  While 

compression  cannot  reduce  the  amount  of  bandwidth  consumed 
by  pre-compressed  files  such  as  MPEG  and  JPEG  formats, 

caching  can. 

Most  communications  officers  are  already  familiar  with 
web  proxies  placed  locally  to  service  the  LAN.  While  this 
technique  does  help,  it  does  not  compare  to  the  benefits  of 
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WAN  caching  techniques  that  utilize  tokens  to  represent  bit 
patterns.  Rather  than  just  locally  caching  HTTP  traffic, 
technologies  such  as  Peribit's  Molecular  Sequence  Reducer 
(MSR)  cache  at  the  bit  level  by  representing  bit  sequences 
with  symbols.  For  example,  an  email  with  a  2  megabyte 
PowerPoint  presentation  may  be  represented  by  a  4  byte 
word.  Rather  than  receiving  the  full  PPT  file,  only  the 
characters  representing  that  file  are  transmitted. 

C.  QUALITY  OF  SERVICE 

A  broad  term  for  bandwidth  management  techniques,  QoS 
refers  to  queuing  strategies,  prioritization,  packet 
shaping/traf f ic  shaping,  policies,  and  flow  control 
designed  to  counter  the  inefficiencies  of  the  internet's 
best  effort  service.  During  periods  of  high  congestion, 
best  effort  service  drops  packets,  requiring  TCP  based 
applications  to  retransmit,  and  offers  no  assurance  that 
application  traffic  is  received.14  As  the  default  service, 
" best  effort  does  not  deny  entry  to  traffic,  so  as  the  load 
levels  increase,  the  network  congestion  levels  increase, 
and  service-quality  levels  decline  uniformly."15  This  can 
become  a  significant  problem  on  tactical  networks,  where 
BERs  can  be  as  high  as  e-516  and  latency  as  high  as 
2000ms.17  "IP  QoS  functions  are  intended  to  deliver 


14  Srinivas  Vegesna,  IP  Quality  of  Service,  p.  5,  Cisco  Press,  2001. 

15  Geoff  Huston,  " Quality  of  Service-Fact  or  Fiction?"  Internet 

Protocol  Journal,  Vol  3,  Number  1,  2000. 

[http : / /www. Cisco . com/warp/public/ 7 5 9/ ipj_3-l/ ipj_3-l_qos . html] .  Date 

not  given,  last  accessed  Feb,  2005. 

15  The  notation  e-5  is  also  represented  as  lxlOA-5,  or  1  error  for 
every  100,000  bits. 

17  Possible  latency  of  EHF  MILSTAR  systems  considering  RTT,  plus 
cross-linking  and  interleaving  processing  time  as  confirmed  by  Mr  Scott 
McCoy,  an  engineer  with  Titan  Corporation  who  supports  MARCORSYSCOM  on 
matters  pertaining  to  EHF  systems . 
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guaranteed  as  well  as  differentiated  Internet  services  by 
giving  network  resource  and  usage  control  to  the  network 
operator . 18 

Unlike  compression,  QoS  features  can  be  implemented  on 
just  one  end  of  the  WAN  to  more  closely  manage  existing 
bandwidth.  The  majority  of  Cisco  devices  come  with  QoS 
features  included  in  the  IOS  (Internetwork  Operating 
System) ,  and  can  be  configured  through  the  Command  Line 
Interface  (CLI) .  However,  the  Marines  are  generally  not 
trained  on  these  techniques.  Recognizing  the  need  to 
optimize  tactical  networks,  I  MEF  took  proactive  measures 
to  prepare  for  their  redeployment  to  Iraq. 

Prior  to  re-deploying  to  OIF  II,  I  MEF  purchased  new 
Cisco  routers  and  layer  3  switches  for  the  MEF  and  its 
MSCs .  During  a  TAD  visit  to  Camp  Pendleton  in  January 
2004,  the  author  observed  the  new  systems  preconfigured  in 
a  lab  environment  to  work  the  bugs  out  and  provide 
additional  training  prior  to  deploying.  Cisco 

representatives  were  present,  walking  the  Marines  through 
configuration  of  QoS  parameters.  However,  by  October  2004, 
the  only  setting  resembling  a  QoS  effort  was  a  VLAN 
dedicating  priority  to  a  VTC  circuit.19  The  router 
administrator  conjectured  that  this  was  due  to 
troubleshooting  by  the  process  of  elimination,  where 
Marines  deleted  configurations  not  absolutely  required. 

Many  PEPs  feature  QoS  settings,  some  of  which  include 
a  GUI  interface,  allowing  administrators  to  easily 
prioritize  traffic  by  protocol,  subnet,  or  VLAN  through 

18  Srinivas  Vegesna,  IP  Quality  of  Service,  p.  5,  Cisco  Press,  2001. 

19  The  author  observed  networks  at  I  MEF,  1st  MarDiv,  RCT  1,  and  2nd 
Battalion/5th  Marines  while  TAD  in  Iraq. 
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check  boxes  or  radio  buttons.  Simplifying  the  interface 
for  configuring  QoS  parameters  and  more  advanced  router 
training  could  help  to  shrink  the  QoS  knowledge  gap  for 
Marines . 

D.  NETWORK  MONITORING  AND  ANALYSIS 

Proactive  network  monitoring  and  analysis  is  an 
effective  way  to  identify  bottlenecks  and  sources  of 
heaviest  traffic.  Monitoring  and  analysis  tools  do  not 
physically  enhance  the  network,  but  rather,  offer  an 
interface  to  the  managers  to  gain  a  better  understanding  of 
what  is  happening  on  the  network  so  that  failures  and 
bottlenecks  can  be  prevented  before  they  occur.  Knowing 
circuit  level  utilization  rates  also  allows  decision-makers 
to  reallocate  unused  bandwidth  to  higher  priority  or  overly 
saturated  links. 

The  requirements  for  effective  network  management  are 
understood  at  all  levels  and  branches  of  service.  During 
Cobra  Gold  2004,  the  Division  G-6  used  a  Commercial  Off  the 
Shelf  (COTS)  solution  called  the  Bandwidth  Management 
Allocation  Controller  (BMAC) ,  a  packet  shaping  and 
Application  layer  QoS  management  system  based  on  Packeteer 
that  can  prioritize  traffic  based  on  protocols.  The  BMAC 
presents  a  good  picture  of  the  network  as  a  whole,  but 
seemed  challenging  to  configure  for  the  one  trained 
operator  present  during  the  exercise.  The  Combined  Joint 
Task  Force  (CJTF)  used  What's  Up  Gold,  while  others  used  HP 
Open  View,  SolarWinds,  and  NTAS .  Each  of  these  programs 
presents  a  slightly  different  and  valuable  picture  of  what 
is  happening  on  the  network.  The  challenge  is  in 
interpretation  of  the  data,  and  standardization  of  which 
system  to  use.  The  solution  to  interpreting  the  data 
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produced  can  be  overcome  through  training.  The  question 
is,  on  which  system  to  train.  Different  units  have 
different  applications ,  thus  seeing  different  pictures  of 
network  health.  Observing  messages  between  the  Army-led 
Joint  Communications  Control  Center  Systems  Control  (JCCC 
SYSCON)  and  the  Combined  Marine  Forces  Systems  Control 
(CMARFOR  SYSCON)  ,  it  became  obvious  that  the  two  were  not 
seeing  the  same  status  of  the  network  due  to  the  different 
applications  being  used,  and  the  means  by  which  that  data 
is  collected,  presented,  and  interpreted. 

The  issue  created  unnecessary  tensions,  and  was  well 
noticed  by  the  staffs  involved.  The  situation  was 

epitomized  by  a  cartoon  which  emerged  during  the  exercise, 
relating  it  to  The  Blind  Men  and  the  Elephant  fable  because 
each  unit  was  seeing  their  specific  piece  of  data,  but 
failing  to  see  and  understand  the  status  of  the  whole 
network.  To  resolve  this  issue,  we  must  move  toward  a 
standardized  application  that  provides  the  best  qualities 
of  all  the  available  tools,  as  well  as  direction  from  the 
Pentagon  Joint  Staff  J-6  to  transition  all  services  to  use 
the  designated  system.  This  concept  is  permeating  all 
branches  of  service  as  the  Joint  Staff  J-6  wrestles  with 
the  fielding  and  implementation  of  the  Joint  Network 
Management  System  (JNMS) . 

E.  PROTOCOL  ENHANCEMENT  PROXIES  (PEPS) 

1 .  The  Problem  with  TCP  Over  Satellite 

TCP  is  a  connection  oriented  protocol,  designed  to 
control  congestion  and  ensure  the  successful  delivery  of 
packets.  The  throughput  of  a  single  TCP  connection  has  a 
maximum  limit  that  is  directly  dependent  upon  the  Bandwidth 
Delay  Product  (BDP)  and  the  Round  Trip  Time  (RTT)  .  The 
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effect  of  a  high  BDP  is  explained  in  a  technical  report 
produced  by  The  Center  for  Satellite  and  Communication 
Networks : 

Consider  as  an  example  a  Tl-rate  (1.544  Mbps)  channel 
through  a  geostationary  satellite,  at  22,300  miles 
altitude.  In  such  a  system,  the  propagation  delay  from 
the  earth's  surface  to  the  satellite  exceeds  120  ms. 
Accordingly,  more  than  120  x  4=480  ms  elapses  between 
the  time  a  byte  is  sent  in  this  system  and  the 
acknowledgment  returns  via  the  satellite.  Multiplying 
this  "roundtrip-time"  by  the  data  transmission  rate 
yields  a  so-called  bandwidth-delay  product  of  more 
than  1544000  x  0.480=741120  bits...  [741  kbps].  The 
bandwidth-delay  product  represents  the  maximum  amount 
of  information  which  can  simultaneously  be  in  transit 
between  the  endpoints  of  the  communication.20 


While  the  previous  example  used  480  ms  to  represent  a 
minimum  RTT,  560  ms  is  more  representative  of  the  DSCS 
satellite  system.  Applying  this  more  toward  a  tactical 
network  scenario,  the  BDP  of  multiplexed  384  Kbps  SIPRNET 
connection  over  a  DSCS  system  would  yield  a  BDP  of  384000 
bps  x  .560  seconds  =  215040  bits  or  215  Kbps,  which  is  only 
56%  of  the  available  bandwidth. 

The  RTT  and  window  size  affects  throughput  based  on 
the  formula: 


throughput  =  window  size  /  RTT 

"Therefore,  using  the  maximum  window  size  of  65,535  bytes 
[64  Kbs]  and  a  geosynchronous  satellite  channel  RTT  of  560 
ms,  the  maximum  throughput  is  limited  to: 


20  M.H.  Hadj itheodosiou,  A.  Ephremides,  D.  Friedman,  The  Center  for 
Satellite  and  Hybrid  Communication  Networks,  CSHCN  T.R.  99-2,  (ISR  T.R. 
99-9)  Broadband  Access  via  Satellite,  p.  21,  Date  not  given. 
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65,535  bytes  /  .560  seconds  =  117,027  bytes/second . "21 


This  means  that  no  matter  how  much  additional  bandwidth  is 
allocated,  throughput  using  DSCS  satellites  is  limited  to 
approximately  117  Kbps  per  TCP  connection.  Applying  the 
throughput  formula  above  to  the  MILSTAR  constellation, 
which  may  have  as  much  as  2000  ms  RTT  due  to  cross-linking, 
processing,  and  interleaving,  the  severely  limited 
capabilities  of  our  SMART-T  links  is  demonstrated:  65,535 
bytes  /  2  seconds  =  32767.5  bytes/second  (32  Kbps). 

No  matter  how  much  bandwidth  is  engineered  into  the  SMART-T 
link,  only  32  Kbps  will  be  utilized  per  TCP  connection. 

It  should  be  noted  that  some  applications  are  capable 
of  multiple  simultaneous  connections,  and  will  compete  for 
more  of  the  available  bandwidth.  This  can  "increase 
throughput  efficiency  to  92%,  with  the  other  8%  accounted 
for  in  overhead...  therefore,  the  effective  window  is 
equal  to  the  sum  of  the  individual  windows."22  However, 
the  number  of  simultaneous  TCP  connections  allowed  is 
unique  to  each  application  and  cannot  be  relied  upon  as  a 
solution  to  the  TCP  over  SATCOM  problem. 

The  reason  BDP  and  RTT  so  adversely  affect  TCP  are  due 
to  the  same  mechanics  designed  to  make  TCP  an  efficient 
layer  4  protocol  over  wired  and  terrestrial  links, 
particularly  the  slow  start  mechanism  and  congestion 
avoidance.  After  transmitting  packets,  TCP  waits  for  an 
Acknowledgement  (ACK)  from  the  distant  end  before 

21  M.  Allman,  D.  Glover,  L.  Sanchez,  Network  Working  Group  Request 
for  Comments:  2488,  Enhancing  TCP  Over  Satellite  Channels  Using 
Standard  Mechanisms,  p.  12,  January,  1999. 

22  Roland  Marc  Selmer,  Internet  Via  Satellite, 

http : // www . selmatex . com/ Publ icat ions/ Interne t %2  QVia%2  0 Satellite . pdf 
Last  accessed  May  1,  2005,  date  of  publication  not  given. 
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transmitting  the  next  burst.  The  size  of  the  segment  that 
can  be  transmitted  is  determined  by  the  congestion  window 
(cwnd) ,  which  is  set  during  the  initial  stages  of  the 
connection.23  Each  time  an  ACK  is  received,  the  cwnd 
increases  by  a  single  segment,  exponentially  increasing  the 
size  of  the  sliding  window  up  to  the  maximum  window  size  of 
64  kbps.24  This  process,  known  as  slow  start,  ensures 
connections  ramp  up  slowly,  rather  than  suddenly  congesting 
the  available  bandwidth.  If  the  ACK  is  not  received  within 
a  specified  time  window,  called  the  Retransmit  Time  Out 
(RTO) ,  TCP  retransmits,  and  the  slow  start  mechanisms 
starts  over.  This  process  was  designed  for  low  latency  / 
low  BER  (10*e~7)  wired  or  terrestrial  networks.  However, 
the  harsh  signal  conditions  of  satellite  links  (10*e~5  BER 
and  500ms+  latency)  adversely  affects  TCP's  slow  start 
congestion  control  algorithm,  causing  it  to  repeatedly  ramp 
up  and  drop.  The  effect  resembles  a  saw  tooth  pattern, 
where  the  area  beneath  the  curve  is  the  average  throughput, 
which  is  significantly  lower  than  the  bandwidth  allotted. 


Figure  4.  Throughput  Average  due  to  TCP  Window  Sizing 

23  M.  Allman  et  al .  Network  Working  Group,  Request  for  Comments: 
2581,  TCP  Congestion  Control,  1999 

24  Roland  Marc  Selmer,  Internet  Via  Satellite, 

http : / /www . selmatex . com/ Publications /Internet%20Via%20 Satellite . pdf 
Last  accessed  May  1,  2005,  date  of  publication  not  given 
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2 .  Using  PEPs  to  Overcome  TCP  Over  SATCOM  Limitations 

Generally,  there  are  two  methods  to  circumvent  the 
problems  of  TCP  over  satellite:  TCP  spoofing,  or  altering 
the  TCP  protocol  itself  into  a  form  more  suited  to  traverse 
the  space  segment.  When  spoofing  is  used,  fake  ACK 
responses  are  returned  locally  to  rapidly  reply  to  SYN 
(synchronization)  and  PSH  (push)  frames,  completing  the 
handshake  and  keeping  the  TCP  window  size  higher. 
Alternatively,  the  open-standard  Space  Communications 
Protocol  Standards  (SCPS)  protocol  is  "essentially  standard 
TCP  augmented  by  a  set  of  extensions  and  enhancements  that 
consist  of  both  implementation  and  specification  changes."25 
The  illustration  below  depicts  the  concept  of  shortening 
ACK  response  times  by  locally  spoofing.26 


Figure  5.  Spoofing  TCP  Acknowledgments  (from  Reference  26) 


25  Darrell  E.  Ernst,  Robert  C.  Durst,  US  Space  Command  (USSPACECOM)  , 
SCPS  D71.51-Y-1,  Space  Communications  Protocol  Standards  (SCPS)  Bent- 
Pipe  Experiment  Report ,  p.  145,  May,  1996. 

26  SFC  Doug  Inglis  WHITE  PAPER:  TCP/IP  Performance  over  Geo-SATCOM 
Links ,  EU32  CONEX  (DISA-EUROPE  STEP  Manager,  date  not  given. 
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Many  PEPs  include  several  of  the  features  identified 
in  the  paragraphs  above,  offering  a  layered  approach  to 
network  optimization.  Therefore,  the  term  PEP  has  become  a 
general  term  for  WAN  acceleration  products,  and  may  do  much 
more  than  just  TCP  enhancement. 

3 .  PEP  Standards  and  Background 

There  are  several  vendors  producing  PEPs  using  both 
the  SCPS  Transport  Protocol  implementation  and  proprietary 
solutions.  Recognizing  the  problems  of  TCP  over  satellite, 
and  the  need  to  provide  the  best  level  of  service  to  the 
warfighters,  DISA  commissioned  defense  contractor  Booz 
Allen  Hamilton  (BAH) ,  along  with  the  MILSATCOM  section  of 
the  Joint  Terminal  Engineering  Office,  to  conduct  an 
analysis  of  several  industry  leading  PEPs.27  PEPs  reviewed 
were  Mentat's  SkyX  Gateway,  Lineo's  VPN  Router,  and  COMSAT 
CLA-2000.  SkyX  was  tested  using  their  original  product 
(Express  Transport  Protocol  (XTP) ) ,  minus  the  compression 
features  instead  of  its  newly  released  SCPS  compatible 
version.  The  test  concept  was  to  transfer  a  single  50  Mb 
file  using  FTP,  across  the  satellite  simulator,  and  assess 
to  what  degree  each  PEP  maximized  the  available  channel.28 
From  the  results  this  and  following  DISA/BAH  PEP  tests  of 
multiple  vendors,  the  SCPS  based  Comtech  Turbo  IP  PEP  was 
chosen  to  be  fielded  at  each  STEP/Teleport . 


27  A  Study  On  Overcoming  TCP/IP  Latency  and  Path  Delays  with  High  Bit 
Error  Rate  Effects  on  Data  Networks  over  Satellite  Communication  Paths, 
MSgt  Robert  Willett,  Sept,  07,2003 

28  Performance  and  Compatibility  Verification  Test  Plan  for  the 
Defense  Information  Systems  Agency  Standardized  Tactical  Entry  Point, 
Booz  Allen  Hamilton  Corporation,  November  18,  2002 
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a .  Interoperabili ty 

Tactical  users  pulling  DISN  services  from  a  STEP 
site  desiring  to  use  PEPs  must  use  the  SCPS  standard. 
Since  DISA's  decision  to  use  the  SCPS  protocol,  several 
competing  PEP  vendors  who  use  proprietary  solutions  have 
developed  and  released  a  SCPS  version  of  their  product.  Of 
particular  interest  are  Expand  and  SkyX,  whose  proprietary 
products  also  feature  the  benefits  of  compression  and/or 
caching.  During  the  Joint  User  Interoperability 
Communications  Exercise  2004  (JUICE  2004),  the  DISA  STEP 
Program  Office  verified  interoperability  of  SkyX' s  SCPS 
version  with  the  Comtech  Turbo  IP  at  the  STEP.29  The  Xiphos 
Xiplink  and  Hotlens  Turbobooster  were  also  verified  as 
compatible.  During  the  PEP  evaluation  conducted  by  the 
author  and  MCTSSA,  Expand' s  new  SCPS  version  was  also 
proven  compatible  with  the  Turbo  IP. 

Many  G-6  representatives  seem  to  be  convinced 
that  they  are  required  to  purchase  the  Comtech  Turbo  IP  for 
STEP  shot  entries,  and  have  the  option  to  purchase  other 
products  for  internal  links.  However,  it  is  not  the  vendor 
mandated  by  DISA,  it  is  the  protocol.  The  Marine  Corps  has 
the  option  of  purchasing  a  device  that  does  both  SCPS  (on 
STEP  entries)  ,  as  well  as  compression,  caching,  and  QoS  on 
internal  links,  all  in  a  single  box.  The  advantages  of 
this  approach  are  significant  in  terms  of  product  support 
and  maintenance,  and  training. 


29  MSgt  Robert  Willet,  Space  Communications  Protocol  Standard  - 
Transport  Protocol  (SCPS-TP)  Interoperability  Testing  for  the  Joint 
User  interoperability  Communications  Exercise  (JUICE)  2004  Test  Report, 
Defense  Information  Systems  Agency,  Satellite  Tactical  Entry  Point 
Program  Office,  Sept  11,  2004 


26 


F.  THE  BEST  SOLUTION:  A  LAYERED  APPROACH 

The  preferred  method  of  maximizing  bandwidth  and 
throughput  is  through  a  layered  approach  that  targets 
several  layers  of  the  OSI  model,  instead  of  just  one. 
The  SCPS  protocol  (layer  4)  may  overcome  TCP  limitations, 
but  can  only  give  approximately  the  full  data  rate  allotted 
to  the  channel,  i.e.,  256  or  384  Kbps.  Even  if  the  maximum 
bandwidth  of  the  channel  is  achieved,  it  remains 
insufficient  for  supporting  the  number  of  users  found  in  a 
Division  COC  or  higher  headquarters.  SCPS  alone  also  does 
nothing  to  enhance  other  transport  protocols,  such  as  the 
growing  percentage  of  UDP  (User  Datagram  Protocol) .  UDP 
supports  applications  such  as  streaming  audio  and  video, 
MPEG  files,  and  Voice  Over  IP  (VoIP)  ,  and  is  becoming  more 
and  more  a  significant  portion  of  the  network  traffic  as 
the  dependency  on  aircraft  and  satellite  imagery  increases. 
An  analysis  of  NTAS  traffic  during  OIF  II,  dated  5  January 
2005,  reveals  UDP  traffic  more  than  doubled  TCP  traffic  on 
an  LMST  link  between  the  MEF  and  the  24th  MEU.30  Using  SCPS 
alone  to  optimize  TCP  would  have  little  bearing  on 
improving  a  link  of  this  nature,  however,  the  addition  of 
compression  would.  A  recent  PEP  test  conducted  for  the 
White  House  Communications  Agency  (WHCA)  by  the  Naval 
Research  Laboratory  demonstrates  the  compressibility  of 
UDP:  "Of  the  devices  tested  with  compression.  Expand 
Accelerator  increased  the  throughput  in  our  UDP  test  from  2 
to  12.4  Mbps,  which  is  a  603%  improvement  in  throughput  of 


30  Author  maintained  the  capability  to  view  MEF' s  SIPRNET  NTAS 
from  the  NPS  STBL  (Secure  Technologies  Battle  Lab)  .  The  date  was 
chosen  as  a  random  sample. 
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the  UDP  data.31  A  layered  combination  of  TCP  enhancement, 
compression,  caching,  QoS,  and  proactive  network  monitoring 
can  greatly  improve  performance  of  the  system  as  a  whole. 

The  simple  diagrams  below  compare  a  non-optimized 
system  with  one  using  a  multilayered  optimization  approach. 
Figure  6  depicts  a  Gigabit  backbone  LAN  becoming 
bottlenecked.  If  data  destined  for  transmission  outside  of 
the  LAN  is  sent  to  the  router  faster  than  it  can  be 
processed,  data  becomes  stacked  in  the  queue.  If  no  QoS  is 
configured,  the  router  directs  packets  using  a  First  In, 
First  Out  (FIFO)  queuing  strategy.  The  data  can  only  be 
processed  as  fast  as  the  allotted  bandwidth  allows,  and  in 
no  order  or  precedence. 

For  comparison.  Figure  7  illustrates  the  benefits  of 
PEPs,  compression,  caching,  and  QoS.  Caching  reduces  the 
amount  of  traffic  required  to  be  sent  over  the  satellite 
link,  and  compression  reduces  the  aggregate  amount  of  data 
before  transmitting.  QoS  policies  ensure  priority  traffic 
is  processed  first,  and  TCP  enhancement  ensures  the  max 
available  bandwidth  is  utilized.  The  result  is  an  opening 
of  the  bottleneck,  creating  a  more  even  flow  of  data. 


31  Naval  Research  Laboratory,  Information  Technology  Division, 
Satellite  and  Wireless  Networking  Section,  Code  5554,  Satellite 
Acceleration  Test  Report,  pg  18,  November  14,  2004. 
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Figure  7.  Multilayered  Optimization  Concept 
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IV.  PEP  EVALUATION 


Figure  8.  PEP  Size  Comparison  to  a  2U  Dell  Server 

A.  INTRODUCTION 

From  16-17  December,  2004,  MCTSSA  hosted  a  technical 

evaluation  of  various  WAN  enhancement  vendors  in  order  to 
provide  data  for  this  thesis,  and  to  validate  a  testing 
strategy  for  future  MCTSSA  tests  of  PEP  technologies. 

1 .  Test  Participants 

Two  of  MCTTSA' s  Systems  Engineers,  Mr.  Paul  Nyenhuis 

and  Mr.  Rickey  Graham,  provided  outstanding  technical 

support  by  validating  test  objectives  and  configuring  the 
lab  and  associated  equipment.  Four  top  PEP  vendors: 

Expand,  Peribit,  Mentat,  and  Comtech,  were  invited  to 
participate  in  the  experiment  based  on  exceptional 
performances  in  previous  similar  evaluations.  All  vendors 
sent  engineers  to  assist,  with  the  exception  of  Comtech, 
which  was  able  to  provide  equipment,  but  not  personnel  due 
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to  previous  commitments.  Figure  8  illustrates  the  size 
comparison  of  each  compared  to  a  2U  Dell  Server  (on 
bottom)  .  The  key  features  of  each  vendor' s  products  are 
represented  in  Table  1 . 


Vendor 

PEP  /  TCP 
Enhancement 

Compression 

Caching 

QoS /Bandwidth 
Management 

Network 
Monitoring 
and  Analysis 

SkyX 

X 

X 

32 

ComTech 

X 

Expand 

X 

X 

X 

X 

X 

Peribit 

X 

X 

X 

X 

X 

Table  1.  Technologies  Featured  by  Vendor. 


2 .  Previous  PEP  Tests 

Although  many  similar  PEP  evaluations  have  been 
conducted,  none  used  a  testing  strategy  that  validated  the 
use  of  SCPS  on  a  network  resembling  the  characteristics  of 
a  USMC  tactical  network.  Most  of  those  reviewed  used  only 
a  single  FTP  session  across  an  unsaturated  satellite  link 
as  the  sample,  and  measured  improvements  SCPS  provided  to 
file  transfer  speed.  This  method  does  not  relate  well  to 
determining  the  benefits  of  PEPs  on  the  typical  highly 

saturated  USMC  tactical  networks .  Only  one  of  the  tests 
reviewed  considered  testing  with  multiple  simultaneous  TCP 
connections.  However,  it  was  only  10  FTP  sessions  from  a 

single  client  across  a  T-l  (1.544  Mbps)  at  750ms  delay  to 
determine  how  various  PEPs  allocated  the  bandwidth.  The 

test  demonstrated  the  SCPS  Transport  Protocol's  ability  to 
fully  utilize  all  the  bandwidth  that  may  go  unused  on  a 

32  SkyX  plans  to  release  Packeteer' s  QoS  features  in  3rd  Quarter, 
2005.  SkyX  (Mentat)  was  purchased  by  Packeteer  in  Dec  2004. 
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normal  TCP  based  link.  However,  providing  all  available 
bandwidth  does  not  help  much  when  the  link  is  98%  consumed. 
For  example,  NTAS  data  from  0200  to  0600,  09  November  2004, 

revealed  a  range  of  91  to  98  percent  utilization  on  one  of 
I  MEF' s  GMF  links.  None  of  the  tests  reviewed  measured  the 
impact  of  SCPS  on  a  highly  saturated,  low  bandwidth  link 
with  multiple  users,  simultaneous  TCP  connections,  and 
multiple  protocols,  mandating  the  conduct  of  this 
evaluation . 

3 .  Purpose 

There  were  several  purposes  for  conducting  this 
evaluation : 

a)  To  assess  performance  improvements  using  a 
test  bed  that  is  more  representative  of  a  USMC  tactical 
network,  where  multiple  simultaneous  TCP  connections, 
users,  and  protocols  saturate  the  satellite  link, 

b)  To  capture  tangible  data  (percentages  of 
improvement)  to  reflect  in  the  optimized  NETWARS  model, 

c)  To  evaluate  interoperability  of  Expand' s  Beta 
release  of  SCPS  with  the  STEP  site's  Comtech  Turbo  IP  PEPs, 

d)  To  validate  MCTSSA' s  testing  configurations 
and  utilities,  such  as  Spirent's  Smartbits  and  VLAN 
configurations  in  preparation  for  a  more  thorough  MCTSSA 
PEP  evaluation  planned  for  the  Spring  of  2005, 

e)  To  determine  if  there  is  an  "all-in-one" 
solution  for  USMC  tactical  WAN/LAN  management. 
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B.  NOTES  ON  THE  TEST  SUBJECTS 
1 .  Comtech  Turbo  IP 

The  Turbo  IP  accelerators  are  the  PEP's  chosen  by  DISA 
to  put  online  at  the  STEP  sites.  For  interoperability 
purposes,  the  Turbo  IP  addresses  only  TCP  enhancement,  and 
features  no  caching,  QoS,  or  compression.  The  web  based 
GUI  proved  simple  to  configure  in  just  a  few  short 
steps . 


Acceleration  Interface 

(&OMTECH 


Routes 


SNMP 


Admin 


rSf  DATABIIffl. 


turboW* 


Acceleration  Active  at  System  Startup  (*  Yes  C  Mo 

Currently  Bypassed  TCP  Port  Numbers 

LAN 

Transmission  Rato  |l  00  bps  C  kbps  Mbps 

WAN 
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Maximum  Round  Trip  Time  (RTT}  in  milliseconds  |ggg 

Maximum  Transfer  Unit  [MTU)  in  bytes  JT500 

I  03131  I  ZOSI  I 


Figure  9.  Comtech  Turbo  IP  GUI  Screenshot 


An  autonegotiation  feature  determines  if  the  distant 
end  is  SCPS  enabled.  If  so,  the  SCPS  implementation  is 
activated.  If  SCPS  is  not  detected,  the  Turbo  IP  reverts 
to  regular  TCP.  This  concept  is  generally  referred  to  as 
Failover  Pass-through . 

2.  Men tat' s  SkyX  Gateway 

All  previously  conducted  tests,  to  include  those 
conducted  by  BAH,  featuring  the  SkyX  Gateway  tested  their 
proprietary  XTP  (Express  Transport  Protocol)  version,  with 

the  exception  of  a  DISA  interoperability  test  conducted 
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shortly  before  this  evaluation.33  The  DISA  test  confirmed 
interoperability  of  Mentat' s  new  beta  version  (5.0)  of  SCPS 
with  the  Turbo  IP.  However,  the  SCPS  beta  version  on  the 
XR-10  model  used  for  this  evaluation  proved  problematic  for 
reasons  undetermined,  but  was  most  likely  due  to  an 
incompatibility  with  the  way  Smartbits  handles  SACKs 
(Selective  Acknowledgments) .  Mentat  made  known  the 

immaturity  of  version  5.0  in  advance,  and  was  willing  to 
support  the  evaluation  for  demonstration  purposes.  Their 
proprietary  version  was  employed  by  I  MEF  during  OIF  I. 

In  December  2004,  one  week  after  the  MCTSSA 
evaluation,  Mentat  was  acquired  by  the  application 
management  company  Packeteer.  In  January,  Packeteer 

announced  the  full  release  of  SkyX  version  5.0,  which 
offers  users  the  capability  to  implement  SCPS  or  XTP  in  a 
single  box.  For  better  performance  on  internal  links, 
units  can  implement  SkyX' s  XTP,  compression,  congestion 
control,  and  Forward  Error  Correction  features,  but  use 
SCPS  on  STEP  shots  or  with  other  SCPS  devices.  When  using 
the  SCPS  only  version,  users  can  still  employ  the  XTP 
congestion  control  capability.  For  compression,  SkyX  uses 
the  open-source  Deflate  algorithm.  Configurations  are  made 
using  the  Command  Line  Interface  (CLI) . 

3 .  Expand 

Expand' s  Accelerator  4800,  version  5.04  was  Expand' s 
first  SCPS  effort.  Their  proprietary  version  was  employed 
during  Exercise  Millennium  Edge  2002,  where  they  "measured 
significantly  higher  data  rates  and  prioritized  services 

33  MSgt  Robert  Willet,  Space  Communications  Protocol  Standard  - 
Transport  Protocol  (SCPS-TP)  Interoperability  Testing  for  the  Joint 
User  interoperability  Communications  Exercise  (JUICE)  2004  Test  Report, 
Defense  Information  Systems  Agency,  Satellite  Tactical  Entry  Point 
Program  Office,  Sept  11,  2004 
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across  existing  WAN  links...  transforming  a  64Kbps  link  into 


a  virtual  256Kbps  link,  and  a  128Kbps  link  into  a  virtual 
512Kbps  link.34  The  24th  MEU  and  I  MEF  also  employed  Expand 
Accelerators  during  OIF  I . 

Expand' s  web  based  GUI  offers  real-time  charts  and 
graphs  for  monitoring  performance,  such  as  depicted  in 
Figure  10 . 


|  ■  P’rc-AccelwMtHJlbpaO  ■  Acceler-ated  (bpa-J  —  Pej*  F>e~A.ccf-ierjflfd  i.bpih  Pe-jl  A.cctlrr-aiwJ  (bpi)  | 


Figure  10.  Expand  Performance  Monitoring  GUI 


4 .  Peribit 

The  SM-500  version  of  Peribit  tested  featured  a  500  Gb 
hard  drive  to  facilitate  is  caching  dictionary.  It  is  SCPS 


34  US  Joint  Forces  Command  Selects  Accelerators  For  Millennium 
Challenge  02  Exercise ,  Case  Study  for  US  Joint  Forces  Command 
http : / / www . expand . com/ include/ casestudy/ JBC . pdf 


Millennium  Challenge  02  Exercise 


36 


compliant,  but  since  compression  cannot  be  disabled,  it 
only  works  with  other  Peribit  systems  and  cannot  be  used 
for  STEP  shot  entries.  The  six  step  configuration  process 
through  Peribit' s  web-based  GUI  proved  quick  and  simple  to 
set  up.  Radio  buttons  allow  configuration  options  such  as 
Forward  Error  Correction,  QoS  features,  and  802. lq  VLAN 


support.  If  vendors  do  not  support  802. lq,  compression 
will  only  occur  on  the  VLAN  on  which  the  Peribit  unit 
resides . 

Peribit  also  features  a  real  time  analysis  GUI  as 
depicted  in  Figure  11. 
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Figure  11. 


Peribit  Performance  Monitoring  GUI 


C .  EXPERIMENTAL  DESIGN 

1.  Physical  Lab  Design: 

The  MCTSSA  testing  lab  is  designed  to  represent  an 
"East  Coast"  LAN  connected  to  a  "West  Coast"  LAN  through  a 
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satellite  simulator.  Spirent's  Smartbits,  an  industry 
leading  network  testing  and  analysis  system,  is  used  to 
generate  multi-protocol  LAN  traffic,  and  to  represent  FTP, 
DNS,  and  WWW  servers  as  well  as  over  100  clients.  Routers 
are  configured  with  VLANs  to  represent  additional  subnets. 

Smartbits  (Avalanche)  was  configured  to  represent  two 
servers  and  multiple  clients  using  multiple,  simultaneous 
FTP,  HTTP,  and  streaming  video  requests.  The  file  sizes  of 
each  of  the  applications  were  tweaked  to  nearly  saturate 
the  512  Kbps  link.  To  assess  the  impact  on  both  satellite 
and  terrestrial  systems,  the  satellite  simulator  was 
configured  to  run  tests  at  both  e-5  and  e-7  BERs .  Tests 
were  first  run  with  0  delay,  then  with  500ms  RTT .  A 
baseline  test  with  no  PEPs  was  run  first  as  a  basis  for 
comparison.  Each  of  the  four  vendors  were  put  through  the 
sequence  of  tests.  To  narrow  the  volumes  of  data 

collected,  and  to  focus  more  on  the  scope  of  this  thesis, 
only  data  from  the  tests  run  at  512  Kbps,  500ms  RTT,  and  e- 
5  BER  is  represented  herein. 

The  following  equipment  was  used,  and  is  represented 
in  the  test  topology  in  Figure  12: 

1  -  Adtech  SX-13A  satellite  simulator 

2  -  Cisco  3745  routers  with  Switchblade 

2  -  Cisco  3750  layer  3  switches 

2  -  SmartBits  with  Smart  Flow  and  Avalanche 

3  -  Laptops  for  SmartBits  consoles  and  HyperTerminal 

2  -  Distributed  Sniffers  for  RMON  capture 
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VLAN  1 ' 


VLAN  43 


Figure  12.  MCTTSA  Lab  Configuration 


D. 


DATA  DESCRIPTIONS 
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Smartbits  collected  statistics  for  each  five  minute 
test  run  in  the  form  of  CSV  files  (Comma-delimited 
Separated  Values) ,  which  were  imported  into  Microsoft 
Excel.  The  raw  data  and  accompanying  graphs  follow. 


BASELINE 

COMTECH  TURBO  IP 

Attempted  Transactions 

198 

Attempted  Transactions 

231 

Successful  Transactions 

184 

Successful  Transactions 

183 

Total  HTTP 

Completions 

164 

Total  HTTP  Completions 

161 

Total  FTP  Completions 

0 

Total  FTP  Completions 

4 

Total  FTP  Attempts 

10 

Total  FTP  Attempts 

10 

Total  Stream 

Completions 

20 

Total  Stream  Completions 

18 

Incomplete  Streams 

4 

Incomplete  Streams 

2 

PERIBIT 

EXPAND  (Proprietary 
Version) 

Attempted  Transactions 

488 

Attempted  Transactions 

344 

Successful  Transactions 

478 

Successful  Transactions 

337 

Total  HTTP 

Completions 

404 

Total  HTTP  Completions 

284 

Total  FTP  Completions 

28 

Total  FTP  Completions 

18 

Total  FTP  Attempts 

28 

Total  FTP  Attempts 

20 

Total  Stream 

Completions 

46 

Total  Stream  Completions 

35 

Incomplete  Streams 

10 

Incomplete  Streams 

5 

Table  2 .  Overview  of  PEP  Test  Results 


No  results  could  be  obtained  for  Mentat's  SKyX  due  to 
technical  problems  with  Smartbits  and  the  beta  versions 
being  tested.  Expand  with  SCPS  demonstrated  similar 
problems  with  the  FTP  portion.  However,  using  only  HTTP 
traffic,  the  SCPS  version  of  Expand  was  verified  to  be 
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compatible  with  the  Turbo  IP.  The  Expand  data  above 
represents  Expand' s  proprietary  solution  on  each  end  for 
demonstrative  purposes. 

E .  ANALYSIS 

The  following  table  illustrates  percentages  of 
improvement  for  the  number  of  successful  transactions: 


% 


Successful  Transactions 

Increase 

Baseline 

184 

Turbo  IP 

183 

-0.54% 

Expand 

337 

83.15% 

Peribit 

478 

159.78% 

Table  3.  Percentage  of  Transactions  Above  Baseline 

All  variables,  such  as  bandwidth,  latency,  BER,  and 
traffic,  were  held  constant  as  each  PEP  was  tested.  The 
more  efficient  the  link  is,  the  more  transactions  TCP  will 
attempt . 
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Figure  13.  Transactions  Completed 

As  expected,  Expand  and  Peribit' s  compression  and 
caching  schemes  (respectively)  allowed  far  more 
transactions  than  SCPS  alone.  According  to  the  data,  the 
SCPS  only  Turbo  IP  demonstrated  no  improvement  over  the 
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baseline.  This  demonstrates  the  inability  of  SCPS  to 
improve  a  link  that  is  saturated  at  maximum  or  near  maximum 
capacity . 

Whereas  the  maximum  throughput  SCPS  could  have 
provided,  whether  the  link  was  congested  or  not,  is  the 
full  512  kbps  of  bandwidth,  the  caching  technology  of 
Peribit  increased  the  perceived  throughput  to  1.2  Mbps.35 


Throughput 
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Figure  14.  Peribit  Acceleration  Rates 


To  put  the  benefit  of  this  improvement  in  perspective 
of  costs,  this  data  rate  is  close  to  the  1.544  Mbps  SIPRNET 
circuits  currently  being  leased  by  I  MEF  in  Iraq.36  I  MEF 
leased  three  8Mb  (aggregate)  Ku  band  terminals  at  a  cost  of 


12  million 

dollars 

per  year. 

Using  Peribit  SM-500s, 

three 

DSCS 

links 

could 

have  been 

optimized  for  a  one 

time 

35 

Figure 

13  and 

Table  4  were 

constructed  using  Peribit' 

s  CSV 

Analyzer,  version  5.0.8. 

36  Costs  for  Ku  band  terminals  provided  by  Captain  Billy  Cornell  who 
deployed  with  I  MEF  G-6  during  OIF  I.  Also  confirmed  by  Lt  Col  Mark 
Bryant,  MARCENT  G-6  Operations  Officer  during  OIF  I. 
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purchase  price  of  about  70,000  dollars,  or  less  than  a  few 
days  worth  of  fees  paid  for  the  leased  Ku  band  terminals. 

The  accelerators  would  also  be  available  for  future 
exercises  and  operations,  where  as  the  leased  terminals  are 
not  owned  by  the  USMC. 

Average 

Accelerated 

Total  Session  Estimated 

TCP  Throughput  Acceleration 

Sessions  Accelerated  Sessions _ (Mbps) _ Factor 


Application 

(count) 

(count) 

(percent) 

(MB) 

Actual 

w/o 

Accel* 

FTP 

56 

54 

96.4% 

27.49 

1.41 

0.24 

5.9  X 

HTTP 

406 

373 

91.9% 

3.37 

0.14 

0.14 

1.0  X 

Others 

46 

40 

87.0% 

0.09 

0.00 

0.00 

1.0  X 

Table  4 .  Peribit  Acceleration  by  Protocol 
It  should  be  noted  that  Peribit' s  caching  method. 


coupled  with  the  way  Smartbits  was  configured  to  send 
redundant  9k  HTTP  files  and  lmb  FTP  files,  may  have  skewed 
the  results.  After  Peribit  caches  data  on  its  500  gigabyte 
hard  drive,  it  represents  that  bit  pattern  with  a  4  byte 

symbol.  Peribit  serves  future  network  requests  matching 

the  cached  data  by  sending  only  the  symbol.  Since  the 
Smartbits  data  was  redundant,  it  is  possible  that  only  4k 
responses  were  traversing  the  WAN  link.  However,  the  data 

pertaining  to  Peribit  and  Expand  is  consistent  with  the 
findings  of  a  November  2004  Network  Computing  magazine  WAN 
acceleration  vendor  assessment.37  Expand  reduced  the 

latency  time  from  950  ms  to  less  than  300  ms,  and  Peribit 
reduced  it  to  around  100  ms.  The  faster  the  ACKs  are 
received,  the  sooner  TCP  can  retransmit  the  next  burst  of 

packets . 

37  Mike  DiMaria,  WAN  Accelerators ,  Breaking  the  NAN  Bottleneck , 
Network  Magazine,  Nov  25,  2004 
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Poor  link  test:  250-ms  round-trip 
latency/lO-ms  jftter/OS%  random  packet  loss 
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Figure  15. 


Network  Computing  Magazine  Test  Results  With 
250ms  RTT  (from  Reference  37) 


Medium-compression  test:  60  Web  pages, 

10  text  and  10  random  files  via  FTP,  20  e-mails; 

50-ms  round-trip  latency/no  jitter/0%  random  packet  loss 
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Figure  16.  Network  Computing  Magazine  Test  Results  With 
Multiple  Files  and  Protocols  (from  Reference  37) 


F .  CONCLUSIONS 

Peribit  clearly  out  performed  the  competition, 
however,  SKY  X  and  Expand  were  testing  beta  software 
versions  implementing  SCPS-TP.  Full  versions  have  since 
been  released.  Although  the  SCPS  version  of  Expand  was  not 
fully  tested,  tests  were  conducted  using  their  proprietary 
version.  Potential  problems  with  Smartbits'  support  of 
Selective  Acknowledgements  (SACK)  are  being  considered  by 
engineers  from  both  Smartbits  and  the  vendors  to  determine 
the  cause.  Therefore,  results  of  this  experiment  are 
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inconclusive  with  respect  to  recommending  a  SCPS  based  PEP 
for  the  Marine  Corps,  but  does  illustrate  the  compounding 
benefits  of  a  layered  optimization  approach  over  the  use  of 
SCPS  alone. 

As  with  any  other  proprietary  solution,  Peribit  could 
be  used  to  optimize  internal  Marine  Corps  links.  The  SM- 
500  has  the  capability  to  recognize  whether  inbound 
circuits  contain  a  like  device.  If  so,  traffic  is 
optimized.  If  not,  it  reverts  to  normal  TCP  methods  for 
that  circuit,  providing  interoperability  with  any  systems 
not  using  Peribit.  However,  because  caching  cannot  be 
disabled  on  Peribit  units,  they  are  not  currently  a  viable 
option  to  interface  with  Comtech  Turbo  IPs  at  the  STEPs. 

Since  this  evaluation,  SkyX  has  announced  the  full 
release  of  their  SCPS  version.  Making  configurations 
changes  through  the  CLI  allows  users  to  implement  SCPS  only 
on  STEP  entries,  or  XTP  with  compression  and  congestion 
control  on  internal  links.  The  compression  features  help 
to  optimize  other  protocols  not  addressed  when  using  SCPS 
alone,  such  as  UDP  traffic. 

G .  RECOMMENDATIONS 

Rather  than  purchasing  Turbo  IP  accelerators  for  STEP 
shots,  and  another  all-in-one  solution  for  internal  links, 
as  organizations  such  as  III  MEF  have  considered, 38  the 
study  demonstrates  that  units  could  purchase  a  single 
product  that  can  do  both.  Doing  so  would  require  less 
training,  costs,  and  support  requirements,  and  offer  more 
flexibility  than  purchasing  two  separate  systems.  DISA's 
decision  to  optimize  the  WAN  link  with  SCPS  focuses  only  on 

38  LtCol  James  M.  Breitinger,  SUB J :  RE:  FW:  Request  for  Information. 
Ill  MEF  intended  to  purchase  Expand  Accelerators  for  internal  links, 
Turbo  IP's  for  STEP  shot  entries. 
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the  satellite  link  and  deprives  tactical  communicators  of 
further  optimizing  their  STEP  shots  with  products  that  have 
better  performance  and  more  features,  such  as  compression, 
caching,  and  QoS.  Even  though  internal  links  can  be 
optimized  as  desired,  traffic  destined  through  the  STEP 
site  will  still  be  constrained  to  the  maximum  bandwidth  of 
the  circuit,  potentially  creating  bottlenecks  for  traffic 
going  through  the  STEP.  Understanding  DISA' s  requirements 
for  open-standards  to  support  interoperability,  perhaps  an 
open-standards  compression  scheme,  such  as  Deflate  in 
conjunction  with  SCPS  is  a  viable  option. 

MCTSSA' s  engineers  intend  to  conduct  follow-on 
testing,  implementing  adjustments  to  the  testing 
methodology  such  as  lab  design  and  adding  actual  clients 
and  servers  to  the  network.  Rather  than  relying  solely  on 
test  tools  to  generate  network  traffic,  actual  clients 
performing  prescribed  tasks  will  produce  more  realistic 
results . 
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V. 


MODELING  AND  SIMULATION  OF  CURRENT  AND 
OPTIMIZED  TACTICAL  SYSTEMS  USING  NETWARS 


A.  INTRODUCTION 

The  Command  and  Control  Research  Program's  (CCRP) , 
Code  of  Best  Practice  for  Experimentation  provides  guidance 
on  producing  usable  models,  reminding  readers  of  George 
E.B.  Fox's  quote  that,  "all  models  are  wrong,  some  are 
useful."39  Although  most  readers  could  ably  deduce  that 
lowering  latency  and  reducing  traffic  loads  will  result  in 
better  network  performance,  this  modeling  and  simulation 
effort  is  intended  to  provide  a  'useful'  statistical  and 
graphical  representation  of  the  improvements. 

1.  About  NETWARS 

An  explanation  of  NETWARS  is  best  given  by  its 
developers,  Booz  Allen  Hamilton: 

NETWARS  is  the  Joint  Standard,  C4  Modeling  and 
Simulation  tool  that  has  been  developed  by  the 
Joint  Staff  J6  in  partnership  with  the  Defense 
Information  Systems  Agency  (DISA)  to  enable  C4 
planners  and  analysts  to  construct  communication 
architectures,  validate  communications  support 
plans,  analyze  existing  and  proposed  network 
architectures,  and  evaluate  the  performance  of 
new  communications  devices  and  applications. 
NETWARS  has  been  developed  to  perform  these 
functions  at  the  strategic,  operational,  and 
tactical  levels  in  support  of  operational 
planners,  analysts,  and  acquisition  communities. 
Output  from  NETWARS  can  be  useful  for  commanders, 
planners,  and  analysts  in  determining  which 
communications  systems  might  be  overloaded  during 
selected  times  in  a  particular  scenario  and  can 
assist  in  making  prudent  architecture  design  and 
acquisition  planning  decisions.40 

39  DOD  Command  and  Control  Research  Program,  Code  of  Best  Practice 
for  Experimentation,  July  2002,  p.401. 

40  NETWARS  Software  Documentation  Manual,  version  2005-1. 
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NETWARS  runs  on  OPNET' s  simulation  engine,  and  is  very 
similar  to  OPNET  with  the  exception  of  being  more 
militarily  focused.  An  OPNET  Simulation  Runtime  license  is 
not  included  in  the  standard  NETWARS  license.  However, 
OPNET  provided  a  temporary  academic  license  to  facilitate 
this  experiment.  The  Simulation  Runtime  license  was 
required  to  actually  run  the  simulations. 

a.  An  Overview  of  NETWARS-  "How  it  Works" 

This  overview  is  to  familiarize  the  reader  with 
the  terminology  used  in  the  proceeding  paragraphs,  and 
provide  a  conceptual  understanding  of  how  a  simulation  is 
constructed  and  analyzed.  Within  NETWARS,  the  top  level 
view  shows  the  Organizations  to  be  represented  in  the 
model.  Within  each  Organization,  Operational  FACilities 
(OPFACS )  are  created  to  represent  collections  of 
communications  devices,  which  virtually  perform  functions 
of  actual  communications  equipment.  The  devices  are 
selected  from  a  library  known  as  the  Object  Pallet,  and 
linked  by  selecting  the  appropriate  link  type.  Data  rates, 
user  profiles,  and  traffic  are  also  configured,  allowing 
'loaded'  traffic  to  be  routed  between  hosts.  The 
simulation  is  then  run  to  produce  the  selected  statistics. 

There  are  many  more  options,  functions,  and 
capabilities  within  NETWARS,  however,  only  those  pertinent 
to  this  scenario  are  introduced  for  the  sake  of  brevity. 

B.  ARCHITECTURE  AND  METHODOLOGY 

NETWARS,  version  2005-1,  is  used  to  demonstrate  in  a 
more  tangible  presentation  the  performance  expectations  of 
a  layered  optimization  strategy  over  the  Marine  Corps' 
current  practices.  Once  the  architecture  is  constructed, 
it  can  be  altered  to  analyze  various  "what-if"  scenarios. 
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The  first  set  of  scenarios  analyzes  current  system 
performance  with  no  optimization.  In  the  second  set  of 
runs,  the  architecture  is  modified  to  represent  the 
improvements  observed  during  the  PEP  evaluation  at  MCTSSA. 

Because  modeling  and  simulation  is  abstract  and 
conceptual  in  nature,  a  simple  architecture  is  used  to 
illustrate  the  concepts  being  presented.  Site  A  and  Site  B 
represent  two  separate  LANs,  such  as  between  the  Division 
and  the  Marine  Air  Wing,  connected  by  a  256k  point-to-point 
WAN  link  typical  of  a  tactical  SIPRNET  or  NIPRNET  circuit. 
The  tests  were  also  run  at  512k  to  demonstrate  the  effects 
on  a  link  with  greater  bandwidth.  Clients  are  connected  to 
the  backside  of  each  router,  and  configured  to  generate 
application  specific  traffic,  i.e.,  the  EMAIL_A  device 
generates  a  high  load  of  SMTP  traffic  across  the  WAN  link. 


Figure  17.  Network  Topology 
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The  delay  on  the  satellite  link  and  the  application 
attributes  are  varied  in  order  to  analyze  system 
performance.  Link  utilization,  link  load  and  application 
download  times  are  used  as  the  performance  metrics  in  all 
comparisons . 

1 .  Traffic 

There  are  several  methods  to  represent  traffic,  such 
as  background  loading  percentages,  the  Application 
Configuration  Utility,  and  importing  actual  traffic 
collected  from  a  network. 

To  import  data  collected  using  Ethereal  or  NTAS,  an  IP 
map  which  maps  every  IP  on  the  system  to  its  source,  must 
be  known.  Most  tactical  networks  contain  multiple  separate 
networks  (SIPRNET,  NIPRNET,  and  often  a  coalition  network) 
each  at  the  MSC  level,  amounting  to  thousands  of  IP 
addresses.  NTAS  data  requires  a  script  to  import,  which 
was  not  available,  and  Ethereal  data  must  be  converted  to 
ACE  threads  to  import.  Due  to  these  issues,  as  well  as 
concerns  with  classification  of  the  data,  this  technique 
was  not  employed  as  originally  intended.  Instead,  the 
traffic  was  constructed  manually  through  the  Application 
Configuration  Utility  based  on  traffic  patterns  observed 
during  UFL  04,  CG04,  and  OIF  II.  NTAS  data  confirmed  the 

top  four  protocols  of  these  networks  as  HTTP,  SMTP,  FTP, 
and  UDP . 

2 .  Modifications  to  Represent  Optimization 

There  is  no  single  number  that  represents  the 
percentage  of  performance  increases  of  a  layered 
optimization  strategy.  There  are  far  too  many  variables 
such  as  protocols,  applications,  and  the  percentages  of 
each  to  generalize  a  "typical"  traffic  profile.  Each  file 
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type  (i.e.,  .doc,  . ppt,  .jpg,  .mpeg,  etc.)  and  protocol 
(i.e.,  HTTP,  FTP,  UDP,  etc.)  varies  in  compressibility. 
Improvements  by  caching  are  dependent  on  the  amount  of 
redundant  data  being  requested.  Improvements  through  SCPS 
depend  on  how  saturated  the  link  may  be,  as  well  as  what 
percentage  of  that  traffic  is  TCP.  UDP  traffic  is  not 
affected  by  SCPS,  but  may  be  compressed  and/or  cached.  To 
represent  compression  and  caching,  the  applications 
traversing  the  network  were  reduced  from  a  heavy  load  to 
light  within  the  Application  Configuration  Utility. 

Because  no  ready-made  PEP  device  exists  in  the  NETWARS 
Object  Pallet,  the  effects  of  PEP  were  represented  by 
adjusting  the  latency  times.  A  standard  250ms  one-way 
delay  represents  a  link  without  PEPs .  To  represent  the 
PEP,  the  delay  was  reduced  to  5ms,  intending  to  resemble 
the  effect  of  a  PEP  producing  the  quality  of  a  terrestrial 
link.  The  traffic  is  configured  at  a  high  load  at  250ms, 
and  a  lighter  load  at  5ms. 

Case  1  at  250ms  represents  the  baseline,  with  no 
compression,  caching,  or  PEP  for  TCP  enhancement.  Cases  2 
and  3  are  omitted  for  brevity  and  relevance.  Cases  4  and  5 
were  configured  with  less  traffic  load  to  represent 
compression  and  caching,  and  latency  reduced  to  5ms  to 
represent  TCP  enhancement.  Cases  4  and  5  differ  only  in 
that  Case  4  omitted  the  VTC  traffic  profile  to  represent 
the  compression  and  caching  of  UDP  traffic,  where  as  Case  5 
includes  the  VTC  profile  to  represent  the  added  traffic  of 
an  actual  VTC  over  IP  session  being  conducted  while  normal 
network  activity  resumes  currently. 

Because  UDP  traffic  is  not  optimized  by  SCPS-TP,  but 

still  comprises  a  percentage  of  the  traffic  traversing  the 
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tactical  networks,  compression  and  caching  must  be  employed 
to  reduce  the  effects  of  UDP.  The  capability  of  using 
compression  and  caching  to  minimize  UDP  traffic  was 
demonstrated  by  the  Naval  Research  Laboratory  PEP 

experiments  in  which  they  experienced  a  600  percent 

reduction  in  perceived  UDP  load  using  compression  and 
caching . 41 
C .  ANALYSIS 


The  following  graphs  produced  in  NETWARS  depict 
margins  of  improvement  for  each  of  the  primary 
applications.  The  graphs  do  not  include  labels  for  the  x 
and  y  axis.  To  clarify  the  graphs,  the  x  axis  shows  the 
progress  of  the  runs  throughout  the  eight  hour  test  period. 
The  y  axis  is  in  seconds. 


Figure  18  demonstrates  the  performance  improvements  to 
TCP  based  applications  using  HTTP  response  times  as  an 
example.  Case  1  (blue)  clearly  shows  slower  response 

times.  Case  4  (red)  demonstrates  improvements  to  response 
times  over  case  5  (green)  when  UDP  traffic  is  reduced  to 
null. 


The  results  consistently  demonstrate  better 
application  response  times  using  a  multilayered 
optimization  approach.  Figure  20  depicts  the  delta  of  case 
5  at  250ms  (no  PEP)  verses  5ms  (with  PEP) .  Other 
application  response  times  are  also  depicted  below. 


44  Naval  Research  Laboratory,  Information  Technology  Division, 
Satellite  and  Wireless  Networking  Section,  Code  5554,  Satellite 
Acceleration  Test  Report,  pg  18,  November  14,  2004 
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Figure  18.  HTTP  Response  Times 


Figure  19.  TCP  Segment  Delay  Reduction 
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average  (in  Video  Confer encing.Packet  End-to-End  Delay 


■  CASE_5_250ms 

■  CASE_5_5ms 

average  (in  Video  Conferencing. Packet  End-to-End  Delay  (sec)) 


Figure  20.  Reduced  VTC  Delay 


Figure  21.  FTP  Response  Times 
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Figure  22 .  Email  Response  Times 
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VI .  CONCLUSIONS 


Given  the  crucial  role  communications  systems  play  in 
information  superiority  and  commanding  and  controlling  the 
battlefield,  bandwidth  has  become  a  critical  asset 
requiring  deliberate  management  techniques.  There  are  many 
optimization  methods  that  could  and  should  be  standard  on 
our  tactical  systems,  but  are  not.  Even  without  PEP's, 
more  effort  could  be  made  to  maximize  the  capabilities  of 
layer  3  devices,  such  as  implementing  purposeful  QoS, 
queuing  strategies,  and  VLANs.  These  features  are  standard 
on  most  Cisco  routers,  but  require  highly  skilled  Marines 
to  plan  and  maintain.  Advanced  training  in  router 

management  techniques  should  be  taught  at  the  Marine  Corps 
Communications  and  Electronics  School  (MCCES)  at  both  an 
administrator  level,  and  a  systems  planner  level.  Router 
optimization  techniques  are  standard  practice  on  corporate 
networks,  where  revenue  is  affected  by  network  performance. 
It  should  also  be  standard  practice  on  tactical  systems, 
where  network  performance  affects  life  or  death  decisions. 

While  we  can  currently  only  employ  SCPS  on  STEP  / 
TELEPORT  entries,  we  can  employ  a  layered  combination  of 
TCP  enhancement,  compression,  caching,  and  QoS  to  fully 
optimize  internal  USMC,  and  perhaps,  with  further  guidance 
from  the  J6  level.  Joint  tactical  systems.  The  MCTSSA 
tests  and  NETWARS  simulation  results  show  that  using  PEPs 
provides  significant  improvements  in  link  utilization  and 
application  response  times  in  comparison  with  non-PEP  uses 
of  TCP  over  a  satellite.  A  multi-function  PEP  that 

includes  compression,  caching,  and  even  QoS  features 
increases  improvement  of  both  TCP  and  UDP  based 
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applications  beyond  the  limits  of  the  apportioned 
bandwidth,  such  as  the  Peribit's  capability  to  make  a  512k 
link  have  nearly  the  equivalent  throughput  of  a  1.544  Mbs 
Tl  line.  A  multilayered  strategy  provides  greater 
perceived  throughput  and  more  efficient  link  utilization  on 
limited  bandwidth  resources. 

Further  experimentation  is  required  to  recommend  a 
specific  product.  Peribit,  Expand,  and  SkyX  with  SCPS 
capabilities  should  be  evaluated  on  an  actual  network,  with 
users  conducting  a  prescribed  testing  agenda. 

Many  G6  Officers  recognize  the  capabilities  of  multi¬ 
purpose  PEPs  and  are  putting  them  online.  However,  units 
rarely  purchase  compatible  devices,  limiting  use  to  their 
own  units.  The  use  of  PEPs  should  be  coordinated  during 
the  Systems  Planning  and  Engineering  phase  at  a  joint  level 
to  fully  maximize  the  potential  benefits.  Until  further 
direction  is  provided,  units  will  continue  to  pursue 
options  that  are  uncoordinated  and  incompatible  with  other 
units.  In  keeping  with  the  ideas  set  forth  in  the  recently 
released  FORCENET  doctrine,  efforts  should  be  made  to 
optimize  our  whole  system  of  systems. 
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